Papers with accelerated inference
On-device Structured and Context Partitioned Projection Networks (P19-1)
Copied to clipboard
| Challenge: | A challenge in on-device text classification is to build highly accurate models that fit in small memory footprint and have low latency. |
| Approach: | They propose an on-device neural network which learns compact projection vectors from raw text using structured and context-dependent partition projections. |
| Outcome: | The proposed model outperforms baseline models and surpasses RNN, CNN and BiLSTM models on dialog act and intent prediction. |
Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field (2022.emnlp-main)
Copied to clipboard
| Challenge: | Entity typing assigns semantic types to entities mentioned in text. |
| Approach: | They propose to use an undirected graphical model to formulate the UFET problem by combining unary potentials with a pairwise conditional random field model. |
| Outcome: | The proposed model outperforms the existing model with little cost and is thousands of times faster than the existing neural network module. |
Scaling Laws and Efficient Inference for Ternary Language Models (2025.acl-long)
Copied to clipboard
Tejas Vaidhya, Ayush Kaushal, Vineet Jain, Francis Couture-Harpin, Prashant Shishodia, Majid Behbahani, Yuriy Nevmyvaka, Irina Rish
| Challenge: | Large language models (LLMs) are increasingly used across research and industry applications, yet their inference efficiency remains a challenge. |
| Approach: | They propose ternary language models that employ quantization-aware training to significantly reduce memory requirements. |
| Outcome: | The proposed ternary language models demonstrate sustained performance gains at scale. |